This invention pertains to a computational method and architecture structure, and more particularly, to such a method and structure which are based on neural-model connectionism.
Recently, the use of the so-called "connectionist" model has gained popularity as an alternative computational paradigm for artificial intelligence systems that display cognitive behavior. Connectionist models are based on the structure of the brain's neural networks, and because of this, are capable of exhibiting computational behavior which is similar to that of the brain. The most important part of such behavior is the ability to process an input and to reach a conclusion in a few steps, instead of the usual thousands of steps which take place in a typical, sequential computer program.
Connectionist models consist of many simple processors that modify the strengths of their interconnections in order to store data. These processing elements in a connectionist network do not solve a given problem individually. Instead, they compute by being connected appropriately to large numbers of similar units. More specifically, they function by generating, in parallel, multiple competing hypotheses, and by then relaxing to a single, best-match interpretation.
In the structure, and according to the method, which are disclosed herein, two types of processors are involved. One of these is referred to as a connection node which, as will be explained, is not a physical element, but rather takes the form of a simple computing "center". The particular function that is computed by a connection node is dependent on the particular connectionist model that has been selected. Implementation of the present invention is in no way dependent upon this selected function, and accordingly, no function discussion is included in the text which follows.
The other kind of processor is referred to as a physical node, which is a physical element that takes the form of an independent processor which is capable of performing standard arithmetic and logical computations. Directly associated with each physical node are plural connection nodes. Stated more precisely, each physical node "creates" (i.e., virtualizes) its associated connection nodes (virtual nodes) through using data stored within the memory structure which also forms part of the physical node.
A principal object of the present invention is to provide a unique architecture structure, and a computational method, which organize and employ physical nodes and connection nodes in a manner that tends to maximize the capabilities and advantages of a neural-model, connectionist, computational network.
More particularly, an object of the invention is to provide such a method and structure that tend to maximize the number of node connections which can exist for communication within the minimum possible space.
Still a further object of the invention is to provide an architecture structure and a method of the types just generally outlined which exhibit a high degree of tolerance to physical flaws or faults in the structure.
According to a preferred embodiment of the invention, what is proposed is referred to herein as a broadcast-hierarchical organization of physical and connection nodes, whereby connection nodes communicate with one another on plural hierarchical, communication (bus) levels. Connection nodes "speak" to one another on the appropriate levels by communicating, inter alia, their local addresses and their "states".
The proposed architecture structure is organized in such a fashion that those connection nodes which communicate with one another most frequently are arranged in closely knit communities, and are connected to "talk" to one another via the lowest-level communication bus, in order to maximize the efficiency of communication. Such localization also minimizes the required memory structure for such nodes by virtue of minimizing the lengths of the addresses which are communicated for identification purposes. The locality of communication in the interconnection architecture proposed herein takes advantage of, and can be tailored to, the natural locality that is a characteristic of connectionist/biological neural network models. Put another way, this organizational concept, which promotes what is referred to herein as locality of communication, essentially assigns the shortest addresses, and therefore the smallest amount of data required to transmit the same, to those connection nodes which experience the greatest frequencies of communication. Nodes with lesser frequencies of communication, which nodes typically communicate on one of the higher communication levels, are assigned longer addresses (a necessary fact which will become apparent), and thus require more data and hence more memory space successfully to communicate these addresses.
A neural, connectionist network of the type generally referred to above will, fundamentally and necessarily, feature extremely large fan-in (inputs) and fan-out (outputs) characteristics vis-a-vis its connection nodes. The broadcast-hierarchical architecture proposed by the present invention is especially suited for accommodating such an environment.
According to a preferred manner of practising the invention, in a broadcast-hierarchical setting including at least two levels of communication, the same includes the steps of establishing a first information-handling level including plural neighborhoods of connection nodes, creating a communication discourse between selected nodes (communication units) on this first level alone whereby a node in a given neighborhood is able to communicate only with a neighboring node, establishing at least a second, higher-level communication level which is effective operatively to link each neighborhood with at least one other neighborhood, and enabling on such second level of communication a communication discourse between a node in a given neighborhood with at least one node in a linked neighborhood.
According to the structure and method outlined above, the various physical nodes and their associated connection nodes operate asynchronously vis-a-vis one another, with communication from a given connection node at a given time broadcast simultaneously to each and every other connection node with which it is expected to communicate. By using the locality of communication concept, which groups and organizes interconnecting connection nodes generally in accordance with their respective frequencies of communications, and by assigning address lengths which generally relate to these frequencies in an inverse proportion, memory space is saved, communication is speeded, and competition for communication bus access is minimized.
Various other features and advantages which are offered by the novel method and apparatus proposed by the present invention will become more fully apparent as the description which now follows is read in conjunction with the drawings.